
<!DOCTYPE html>
<html>
  <head>
    
<meta charset="utf-8" >

<title>mxnet gluon  | dragon</title>
<meta name="description" content="邮箱(base64)：MTY5MDMwMjk2M0BxcS5jb20=
">

<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/animate.css/3.7.0/animate.min.css">

<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.7.2/css/all.css" integrity="sha384-fnmOCqbTlWIlj8LyTjo7mOUStjsKC4pOpQbqyi7RrhN7udi9RwhKkMHpvLbHG9Sr" crossorigin="anonymous">
<link rel="shortcut icon" href="https://dragonfive.gitee.io//favicon.ico?v=1740893463017">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.10.0/katex.min.css">
<link rel="stylesheet" href="https://dragonfive.gitee.io//styles/main.css">



<script src="https://cdn.jsdelivr.net/npm/vue/dist/vue.js"></script>
<script src="//cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.5.1/build/highlight.min.js"></script>



  </head>
  <body>
    <div id="app" class="main">
      <div class="site-header-container">
  <div class="site-header">
    <div class="left">
      <a href="https://dragonfive.gitee.io/">
        <img class="avatar" src="https://dragonfive.gitee.io//images/avatar.png?v=1740893463017" alt="" width="32px" height="32px">
      </a>
      <a href="https://dragonfive.gitee.io/">
        <h1 class="site-title">dragon</h1>
      </a>
    </div>
    <div class="right">
      <transition name="fade">
        <i class="icon" :class="{ 'icon-close-outline': menuVisible, 'icon-menu-outline': !menuVisible }" @click="menuVisible = !menuVisible"></i>
      </transition>
    </div>
  </div>
</div>

<transition name="fade">
  <div class="menu-container" style="display: none;" v-show="menuVisible">
    <div class="menu-list">
      
        
          <a href="/" class="menu purple-link">
            首页
          </a>
        
      
        
          <a href="/archives" class="menu purple-link">
            归档
          </a>
        
      
        
          <a href="/tags" class="menu purple-link">
            标签
          </a>
        
      
        
          <a href="/post/about" class="menu purple-link">
            关于
          </a>
        
      
    </div>
  </div>
</transition>


      <div class="content-container">
        <div class="post-detail">
          
          <h2 class="post-title">mxnet gluon </h2>
          <div class="post-info post-detail-info">
            <span><i class="icon-calendar-outline"></i> 2018-03-22</span>
            
              <span>
                <i class="icon-pricetags-outline"></i>
                
                  <a href="https://dragonfive.gitee.io/tag/lz8Owz8ML/">
                    gluon
                    
                      ，
                    
                  </a>
                
                  <a href="https://dragonfive.gitee.io/tag/MLJqs9GOQ/">
                    networker
                    
                      ，
                    
                  </a>
                
                  <a href="https://dragonfive.gitee.io/tag/o9zBQ9B4NY/">
                    神经网络
                    
                      ，
                    
                  </a>
                
                  <a href="https://dragonfive.gitee.io/tag/WzibKNMac/">
                    深度学习
                    
                  </a>
                
              </span>
            
          </div>
          <div class="post-content" v-pre>
            <hr>
<p>title: gluon学习笔记<br>
date: 2018/3/22 12:04:12<br>
categories:</p>
<ul>
<li>深度学习<br>
tags:</li>
<li>目标检测</li>
<li>深度学习</li>
<li>神经网络</li>
</ul>
<hr>
<h1 id="学到的新知识">学到的新知识</h1>
<h2 id="bn放在relu后面">bn放在relu后面</h2>
<p><a href="http://minibatch.net/2017/06/11/%E7%BB%8F%E5%85%B8%E8%AE%BA%E6%96%87-Batch-Normalization/">BN应该放在relu后</a></p>
<p><a href="https://mp.weixin.qq.com/s/xJromD5Q30KlRhB_kM4kfA">用于分类、检测和分割的移动网络 MobileNetV2</a></p>
<p><a href="https://www.zhihu.com/question/265709710">如何评价MobileNetV2</a></p>
<h2 id="卷积核的数量">卷积核的数量</h2>
<p><a href="http://zh.gluon.ai/chapter_convolutional-neural-networks/cnn-scratch.html">卷积神经网络 — 从0开始</a></p>
<p>当输入数据有多个通道的时候，每个通道会有对应的权重，然后会对每个通道做卷积之后在通道之间求和。所以当输出只有一个的时候，卷积的channel数目和data的channel数目是一样的。</p>
<p>当输出需要多通道时，每个输出通道有对应权重，然后每个通道上做卷积。所以当输入有n个channel，输出有h个channel时，卷积核channel数目为n * h，每个输出channel对应一个bias ,卷积核的维度为(h,n,w,h)</p>
<p class='katex-block'><span class="katex-display"><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>c</mi><mi>o</mi><mi>n</mi><mi>v</mi><mo>(</mo><mi>d</mi><mi>a</mi><mi>t</mi><mi>a</mi><mo separator="true">,</mo><mi>w</mi><mo separator="true">,</mo><mi>b</mi><mo>)</mo><mo>[</mo><mo>:</mo><mo separator="true">,</mo><mi>i</mi><mo separator="true">,</mo><mo>:</mo><mo separator="true">,</mo><mo>:</mo><mo>]</mo><mo>=</mo><mi>c</mi><mi>o</mi><mi>n</mi><mi>v</mi><mo>(</mo><mi>d</mi><mi>a</mi><mi>t</mi><mi>a</mi><mo separator="true">,</mo><mi>w</mi><mo>[</mo><mi>i</mi><mo separator="true">,</mo><mo>:</mo><mo separator="true">,</mo><mo>:</mo><mo separator="true">,</mo><mo>:</mo><mo>]</mo><mo separator="true">,</mo><mi>b</mi><mo>[</mo><mi>i</mi><mo>]</mo><mo>)</mo></mrow><annotation encoding="application/x-tex">conv(data, w, b)[:,i,:,:] = conv(data, w[i,:,:,:], b[i])
</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathdefault">c</span><span class="mord mathdefault">o</span><span class="mord mathdefault">n</span><span class="mord mathdefault" style="margin-right:0.03588em;">v</span><span class="mopen">(</span><span class="mord mathdefault">d</span><span class="mord mathdefault">a</span><span class="mord mathdefault">t</span><span class="mord mathdefault">a</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord mathdefault" style="margin-right:0.02691em;">w</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord mathdefault">b</span><span class="mclose">)</span><span class="mopen">[</span><span class="mrel">:</span></span><span class="base"><span class="strut" style="height:0.85396em;vertical-align:-0.19444em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord mathdefault">i</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">:</span></span><span class="base"><span class="strut" style="height:0.625em;vertical-align:-0.19444em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">:</span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mclose">]</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathdefault">c</span><span class="mord mathdefault">o</span><span class="mord mathdefault">n</span><span class="mord mathdefault" style="margin-right:0.03588em;">v</span><span class="mopen">(</span><span class="mord mathdefault">d</span><span class="mord mathdefault">a</span><span class="mord mathdefault">t</span><span class="mord mathdefault">a</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord mathdefault" style="margin-right:0.02691em;">w</span><span class="mopen">[</span><span class="mord mathdefault">i</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">:</span></span><span class="base"><span class="strut" style="height:0.625em;vertical-align:-0.19444em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">:</span></span><span class="base"><span class="strut" style="height:0.625em;vertical-align:-0.19444em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">:</span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mclose">]</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord mathdefault">b</span><span class="mopen">[</span><span class="mord mathdefault">i</span><span class="mclose">]</span><span class="mclose">)</span></span></span></span></span></p>
<p class='katex-block'><span class="katex-display"><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi mathvariant="normal">Γ</mi><mo>(</mo><mi>z</mi><mo>)</mo><mo>=</mo><msubsup><mo>∫</mo><mn>0</mn><mi mathvariant="normal">∞</mi></msubsup><msup><mi>t</mi><mrow><mi>z</mi><mo>−</mo><mn>1</mn></mrow></msup><msup><mi>e</mi><mrow><mo>−</mo><mi>t</mi></mrow></msup><mi>d</mi><mi>t</mi><mtext> </mtext><mi mathvariant="normal">.</mi></mrow><annotation encoding="application/x-tex">\Gamma(z) = \int_0^\infty t^{z-1}e^{-t}dt\,.
</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">Γ</span><span class="mopen">(</span><span class="mord mathdefault" style="margin-right:0.04398em;">z</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:2.326242em;vertical-align:-0.9119499999999999em;"></span><span class="mop"><span class="mop op-symbol large-op" style="margin-right:0.44445em;position:relative;top:-0.0011249999999999316em;">∫</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.414292em;"><span style="top:-1.7880500000000001em;margin-left:-0.44445em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span><span style="top:-3.8129000000000004em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">∞</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.9119499999999999em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord mathdefault">t</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.864108em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right:0.04398em;">z</span><span class="mbin mtight">−</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathdefault">e</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.843556em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mathdefault mtight">t</span></span></span></span></span></span></span></span></span><span class="mord mathdefault">d</span><span class="mord mathdefault">t</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord">.</span></span></span></span></span></p>
<p>123</p>
<figure data-type="image" tabindex="1"><img src="https://www.github.com/DragonFive/CVBasicOp/raw/master/1513949196873.jpg" alt="inception v1" loading="lazy"></figure>
<figure data-type="image" tabindex="2"><img src="https://www.github.com/DragonFive/CVBasicOp/raw/master/1514013854016.jpg" alt="residual" loading="lazy"></figure>
<figure data-type="image" tabindex="3"><img src="https://www.github.com/DragonFive/CVBasicOp/raw/master/1514012389756.jpg" alt="resnet各种结构" loading="lazy"></figure>
<h1 id="gluon语法">gluon语法</h1>
<h2 id="nnblock与nnsequential的嵌套使用">nn.Block与nn.sequential的嵌套使用</h2>
<pre><code class="language-python">class RecMLP(nn.Block):
    def __init__(self, **kwargs):
        super(RecMLP, self).__init__(**kwargs)
        self.net = nn.Sequential()
        with self.name_scope():
            self.net.add(nn.Dense(256, activation=&quot;relu&quot;))
            self.net.add(nn.Dense(128, activation=&quot;relu&quot;))
            self.dense = nn.Dense(64)

    def forward(self, x):
        return nd.relu(self.dense(self.net(x)))

rec_mlp = nn.Sequential()
rec_mlp.add(RecMLP())
rec_mlp.add(nn.Dense(10))
print(rec_mlp)
</code></pre>
<h2 id="初始化与参数访问">初始化与参数访问</h2>
<pre><code class="language-python">from mxnet import init
params.initialize(init=init.Normal(sigma=0.02), force_reinit=True)
print(net[0].weight.data(), net[0].bias.data())
</code></pre>
<p>我们也可以通过collect_params来访问Block里面所有的参数（这个会包括所有的子Block）。它会返回一个名字到对应Parameter的dict。</p>
<p>也可以自定义各层的初始化方法，没有自定义的按照net.initialize里面的方法进行定义</p>
<pre><code class="language-python">from mxnet.gluon import nn
from mxnet import nd
from mxnet import init

def get_net():
    net = nn.Sequential()
    with net.name_scope():
        net.add(nn.Dense(4,activation=&quot;relu&quot;))#,weight_initializer=init.Xavier()))
        net.add(nn.Dense(2,weight_initializer=init.Zero(),bias_initializer=init.Zero()) )
    return net

x = nd.random.uniform(shape=(3,5))
net = get_net()
net.initialize(init.One())
net(x)
print(net[1].weight.data
</code></pre>
<h2 id="gpu访问">GPU访问</h2>
<ol>
<li>删除cpu版本mxnet</li>
</ol>
<pre><code class="language-bash">pip uninstall mxnet
</code></pre>
<ol start="2">
<li>更新GPU版本mxnet</li>
</ol>
<pre><code class="language-bash">pip install -U --pre mxnet-cu80
</code></pre>
<ol start="3">
<li>查看版本号</li>
</ol>
<pre><code class="language-python">import pip
for pkg in ['mxnet', 'mxnet-cu75', 'mxnet-cu80']:
    pip.main(['show', pkg])
</code></pre>
<h2 id="使用jupyter的相关插件">使用jupyter的相关插件</h2>
<ol>
<li>notedown插件<br>
可以在jupyter 中查看markdown文件</li>
<li>nb_conda<br>
是conda的插件，可以在jupyter里面修改python内核版本</li>
</ol>
<h2 id="优化方法">优化方法</h2>
<p><strong>momentum</strong><br>
gluon.Trainer的learning_rate属性和set_learning_rate函数可以随意调整学习率。</p>
<pre><code class="language-python">trainer = gluon.Trainer(net.collect_params(), 'sgd',
                            {'learning_rate': lr, 'momentum': mom})
</code></pre>
<p><strong>adagrad</strong><br>
Adagrad是一个在迭代过程中不断自我调整学习率，并让模型参数中每个元素都使用不同学习率的优化算法。</p>
<pre><code class="language-python">    trainer = gluon.Trainer(net.collect_params(), 'adagrad',
                            {'learning_rate': lr})
</code></pre>
<p><strong>Adam</strong></p>
<pre><code class="language-python">trainer = gluon.Trainer(net.collect_params(), 'adam',
                            {'learning_rate': lr})

</code></pre>
<p>通过以上分析, 理论上可以说, 在数据比较稀疏的时候, adaptive 的方法能得到更好的效果, 例如, adagrad, adadelta, rmsprop, adam 等. 在数据稀疏的情况下, adam 方法也会比 rmsprop 方法收敛的结果要好一些, 所以, 通常在没有其它更好的理由的前框下, 我会选用 adam 方法, 可以比较快地得到一个预估结果. 但是, 在论文中, 我们看到的大部分还是最原始的 mini-batch 的 SGD 方法. 因为马鞍面的存在等问题, SGD 方法有时候较难收敛. 另外, SGD 对于参数的初始化要求也比较高. 所以, 如果要是想快速收敛的话, 建议使用 adam 这类 adaptive 的方法</p>
<h2 id="延迟执行">延迟执行</h2>
<p>延后执行使得系统有更多空间来做性能优化。但我们推荐每个批量里至少有一个同步函数，例如对损失函数进行评估，来避免将过多任务同时丢进后端系统。</p>
<pre><code class="language-python">from mxnet import autograd

mem = get_mem()

total_loss = 0
for x, y in get_data():
    with autograd.record():
        L = loss(y, net(x))
    total_loss += L.sum().asscalar()
    L.backward()
    trainer.step(x.shape[0])

nd.waitall()
print('Increased memory %f MB' % (get_mem() - mem))

</code></pre>
<h2 id="多gpu训练">多GPU训练</h2>
<pre><code class="language-python">ctx = [gpu(i) for i in range(num_gpus)]
data_list = gluon.utils.split_and_load(data, ctx)
label_list = gluon.utils.split_and_load(label, ctx)



</code></pre>
<h2 id="fintune-微调">fintune 微调</h2>
<p><a href="https://fiercex.github.io/post/gluon_features_fine/">gluon微调</a></p>
<h1 id="一些可以重复使用的代码">一些可以重复使用的代码</h1>
<h2 id="读取数据">读取数据</h2>
<pre><code class="language-python">from mxnet import gluon
from mxnet import ndarray as nd

def transform(data, label):
    return data.astype('float32')/255, label.astype('float32')
mnist_train = gluon.data.vision.FashionMNIST(train=True, transform=transform)
mnist_test = gluon.data.vision.FashionMNIST(train=False, transform=transform)

</code></pre>
<h2 id="计算精度">计算精度</h2>
<pre><code class="language-python">def accuracy(output, label):
    return nd.mean(output.argmax(axis=1)==label).asscalar()


</code></pre>
<p>我们先使用Flatten层将输入数据转成 batch_size x ? 的矩阵，然后输入到10个输出节点的全连接层。照例我们不需要制定每层输入的大小，gluon会做自动推导。</p>
<h2 id="激活函数">激活函数</h2>
<p><strong>sigmoid</strong></p>
<pre><code class="language-python">from mxnet import nd
def softmax(X):
    exp = nd.exp(X)
    # 假设exp是矩阵，这里对行进行求和，并要求保留axis 1，
    # 就是返回 (nrows, 1) 形状的矩阵
    partition = exp.sum(axis=1, keepdims=True)
    return exp / partition


</code></pre>
<p><strong>relu</strong></p>
<pre><code class="language-python">def relu(X):
    return nd.maximum(X, 0)

</code></pre>
<h2 id="损失函数">损失函数</h2>
<p><strong>平方误差</strong></p>
<pre><code class="language-python">square_loss = gluon.loss.L2Loss()


</code></pre>
<pre><code class="language-python">def square_loss(yhat, y):
    # 注意这里我们把y变形成yhat的形状来避免矩阵形状的自动转换
    return (yhat - y.reshape(yhat.shape)) ** 2
 

</code></pre>
<p><strong>交叉熵损失</strong></p>
<pre><code class="language-python">def cross_entropy(yhat, y):
    return - nd.pick(nd.log(yhat), y)

</code></pre>
<pre><code class="language-python">softmax_cross_entropy = gluon.loss.SoftmaxCrossEntropyLoss()

</code></pre>
<h2 id="取一个batch_size的代码">取一个batch_size的代码</h2>
<p><strong>scratch版本</strong></p>
<pre><code class="language-python">import random
batch_size = 1
def data_iter(num_examples):
    idx = list(range(num_examples))
    random.shuffle(idx)
    for i in range(0, num_examples, batch_size):
        j = nd.array(idx[i:min(i+batch_size,num_examples)])
        yield X.take(j), y.take(j)

</code></pre>
<p><strong>gluon版本</strong></p>
<pre><code class="language-python">batch_size = 1
dataset_train = gluon.data.ArrayDataset(X_train, y_train)
data_iter_train = gluon.data.DataLoader(dataset_train, batch_size, shuffle=True)



</code></pre>
<h2 id="初始化权值">初始化权值</h2>
<p><strong>scratch版本</strong></p>
<pre><code class="language-python">def get_params():
    w = nd.random.normal(shape=(num_inputs, 1))*0.1
    b = nd.zeros((1,))
    for param in (w, b):
        param.attach_grad()
    return (w, b)

</code></pre>
<p><strong>gluon版本</strong></p>
<pre><code class="language-python">net.initialize()


net.collect_params().initialize(mx.init.Normal(sigma=1))

</code></pre>
<h2 id="sgd">SGD</h2>
<p><strong>scratch版本</strong></p>
<pre><code class="language-python">def SGD(params, lr):
    for param in params:
        param[:] = param - lr * param.grad


</code></pre>
<p>L2正则</p>
<pre><code class="language-python">def L2_penalty(w, b):
    return ((w**2).sum() + b**2) / 2

</code></pre>
<p><strong>gluon版本</strong></p>
<pre><code>    trainer = gluon.Trainer(net.collect_params(), 'sgd', {
        'learning_rate': learning_rate, 'wd': weight_decay})

</code></pre>
<p>这里的weight_decay表明这里添加了L2正则，正则化<br>
w = w -lr * grad - wd * w</p>
<h2 id="训练过程">训练过程</h2>
<p><strong>scratch版本</strong></p>
<pre><code class="language-python">    for e in range(epochs):        
        for data, label in data_iter(num_train):
            with autograd.record():
                output = net(data, lambd, *params)
                loss = square_loss(
                    output, label) + lambd * L2_penalty(*params)
            loss.backward()
            SGD(params, learning_rate)
        train_loss.append(test(params, X_train, y_train))
        test_loss.append(test(params, X_test, y_test))

</code></pre>
<p><strong>gluon版本</strong></p>
<pre><code class="language-python">    for e in range(epochs):        
        for data, label in data_iter_train:
            with autograd.record():
                output = net(data)
                loss = square_loss(output, label)
            loss.backward()
            trainer.step(batch_size)            
        train_loss.append(test(net, X_train, y_train))
        test_loss.append(test(net, X_test, y_test))


</code></pre>
<pre><code class="language-python">%matplotlib inline
import matplotlib as mpl
mpl.rcParams['figure.dpi']= 120
import matplotlib.pyplot as plt

def train(X_train, X_test, y_train, y_test):
    # 线性回归模型
    net = gluon.nn.Sequential()
    with net.name_scope():
        net.add(gluon.nn.Dense(1))
    net.initialize()
    # 设一些默认参数
    learning_rate = 0.01
    epochs = 100
    batch_size = min(10, y_train.shape[0])
    dataset_train = gluon.data.ArrayDataset(X_train, y_train)
    data_iter_train = gluon.data.DataLoader(
        dataset_train, batch_size, shuffle=True)
    # 默认SGD和均方误差
    trainer = gluon.Trainer(net.collect_params(), 'sgd', {
        'learning_rate': learning_rate})
    square_loss = gluon.loss.L2Loss()
    # 保存训练和测试损失
    train_loss = []
    test_loss = []
    for e in range(epochs):
        for data, label in data_iter_train:
            with autograd.record():
                output = net(data)
                loss = square_loss(output, label)
            loss.backward()
            trainer.step(batch_size)
        train_loss.append(square_loss(
            net(X_train), y_train).mean().asscalar())
        test_loss.append(square_loss(
            net(X_test), y_test).mean().asscalar())
    # 打印结果
    plt.plot(train_loss)
    plt.plot(test_loss)
    plt.legend(['train','test'])
    plt.show()
    return ('learned weight', net[0].weight.data(),
            'learned bias', net[0].bias.data())

</code></pre>
<p>最终版</p>
<pre><code class="language-python">def train(train_data, test_data, net, loss, trainer, ctx, num_epochs, print_batches=None):
    &quot;&quot;&quot;Train a network&quot;&quot;&quot;
    print(&quot;Start training on &quot;, ctx)
    if isinstance(ctx, mx.Context):
        ctx = [ctx]
    for epoch in range(num_epochs):
        train_loss, train_acc, n, m = 0.0, 0.0, 0.0, 0.0
        if isinstance(train_data, mx.io.MXDataIter):
            train_data.reset()
        start = time()
        for i, batch in enumerate(train_data):
            data, label, batch_size = _get_batch(batch, ctx)
            losses = []
            with autograd.record():
                outputs = [net(X) for X in data]
                losses = [loss(yhat, y) for yhat, y in zip(outputs, label)]
            for l in losses:
                l.backward()
            train_acc += sum([(yhat.argmax(axis=1)==y).sum().asscalar()
                              for yhat, y in zip(outputs, label)])
            train_loss += sum([l.sum().asscalar() for l in losses])
            trainer.step(batch_size)
            n += batch_size
            m += sum([y.size for y in label])
            if print_batches and (i+1) % print_batches == 0:
                print(&quot;Batch %d. Loss: %f, Train acc %f&quot; % (
                    n, train_loss/n, train_acc/m
                ))

        test_acc = evaluate_accuracy(test_data, net, ctx)
        print(&quot;Epoch %d. Loss: %.3f, Train acc %.2f, Test acc %.2f, Time %.1f sec&quot; % (
            epoch, train_loss/n, train_acc/m, test_acc, time() - start
        ))

</code></pre>
<h1 id="reference">reference</h1>
<p><a href="https://zhuanlan.zhihu.com/p/28867241">从零开始码一个皮卡丘检测器</a></p>
<p><a href="http://blog.csdn.net/jesse_mx/article/details/53606897">图片标注工具</a></p>
<p><a href="http://blog.csdn.net/u014696921/article/details/56877979"> mxnet 使用自己的图片数据训练CNN模型</a></p>
<p><a href="https://mxnet.incubator.apache.org/api/python/image.html#Image">mxnet image API</a></p>
<p><a href="https://mxnet.incubator.apache.org/how_to/recordio.html?highlight=recordio">Create a Dataset Using RecordIO</a></p>
<p><a href="http://blog.csdn.net/muyouhang/article/details/77727381">基于MXNet gluon 的SSD模型训练</a></p>
<p><a href="https://groups.google.com/a/continuum.io/forum/m/#!topic/anaconda/RuSpZVPEio8">解决conda与ipython notebook的python版本问题</a></p>
<p><a href="http://blog.csdn.net/sunshine_in_moon/article/details/51434908">神经网络计算参数量的方法</a></p>
<p><a href="https://www.jianshu.com/p/c56a37093cfa">神经网络计算特征图的大小的方法</a></p>
<p><a href="http://minibatch.net/2017/06/11/%E7%BB%8F%E5%85%B8%E8%AE%BA%E6%96%87-Batch-Normalization/">BN应该放在relu后</a></p>

          </div>
        </div>

        
          <div class="next-post">
            <a class="purple-link" href="https://dragonfive.gitee.io/post/cpp-keyword/">
              <h3 class="post-title">
                下一篇：c++ 的关键字回顾
              </h3>
            </a>
          </div>
          
      </div>

      

      <div class="site-footer">
  <div class="slogan">邮箱(base64)：MTY5MDMwMjk2M0BxcS5jb20=
</div>
  <div class="social-container">
    
      
        <a href="https://github.com/DragonFive" target="_blank">
          <i class="fab fa-github"></i>
        </a>
      
    
      
    
      
    
      
    
      
    
  </div>
  Powered by <a href="https://github.com/getgridea/gridea" target="_blank">Gridea</a> | <a class="rss" href="https://dragonfive.gitee.io//atom.xml" target="_blank">RSS</a>
</div>


    </div>
    <script type="application/javascript">

hljs.initHighlightingOnLoad()

var app = new Vue({
  el: '#app',
  data: {
    menuVisible: false,
  },
})

</script>




  </body>
</html>
